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1. INTRODUCTION 

Qwerty Bank is a limited liability company engaged in the financial services industry. Qwerty Bank 
focuses on the micro, small and medium enterprises (MSME) segment. Currently, it has grown and 
developed into a bank that has entered the group of medium-sized banks in Indonesia in terms of assets. 
Along with opening up opportunities and increasing the ability to serve the needs of a wider community, 
Qwerty Bank has expanded its business to the commercial and consumer segments. These three segments are 
the business pillars of Qwerty Bank, with services supported by an optimal fund management system, 
reliable information technology, competence in human resources and good corporate governance practices. 
This foundation allowed Qwerty Bank to step forward and position it as a credible bank. The operation of 
Qwerty Bank is currently supported by more than 425 outlets spread across 22 provinces throughout 
Indonesia which are connected in real time. Qwerty Bank has also built a micro-banking network, which now 
totals 543 outlets, as part of a partnership program with cooperatives and microfinance institutions. 

Customer satisfaction and trust in using bank products and services will be able to attract the interest 
of prospective new customers [1]. Optimal service certainly provides a positive image for the company 


Journal homepage: http://ijai.iaescore.com 


24 im) ISSN: 2252-8938 


thereby increasing the company's image in the eyes of customers and prospective customers. One of the roles 
is done by officers who deal directly with customers, including marketing, customer service and call centers. 
Functionally, customer service and call center have the same role, namely as a company representative to 
interact with customers. Broadly speaking, the difference can be seen from the media used to interact, where 
customer service uses face-to-face media, while the call center uses telephone media. 

One of the services provided by Qwerty Bank to customers is call center service. About 50% of the 
top 18 customer services are information services. Call Center services have 181 service categories that vary 
in both banking and credit cards. The services used at Qwerty Bank are using Halo Qwerty 14555 which is 
managed by a third party where the bank has to pay for such a service based on a fixed scheme every time the 
number of calls is less than 150,000. However, if there are more than 150,000 calls per month, the cost will 
be 2,750 per call. 

There is a trend of decreasing number of traffic calls from year to year, but this is not directly 
proportional to the costs incurred for the Halo Qwerty call center service 14555 every year. One of them can 
be seen in 2018, the average number of call center services has decreased significantly. When viewed in 
terms of service, this can be interpreted to be a positive thing for the company, because it indicates the 
reduction in the number of customer complaints against Qwerty Bank services. However, this is inversely 
proportional in terms of cost. With an average of 115,776 calls per month, the costs incurred for the call 
center facility every month are Rp 625,000,000.00, or in a year around Rp 7,500,000,000.00. When seen the 
cost per call, it will increasingly look more expensive every year when compared to the cost per call 
Rp 2,750.00 if the number of calls reaches 150,000 every month. For example, in 2017, the cost per call is 
Rp 2,750.00, while in 2018 the cost per call will be Rp 5,389.35. 

With the increasing use of Chatbot technology, more and more Chatbot platforms are being 
developed. This can be shown by the large number of information technology (IT) companies that focus their 
business on the field of Chatbot, such as Kata.ai (https://kata.ai), Rasa Platform (https://rasa.com), Botika 
(https://botika.online), and so on. However, there is not yet one Chatbot platform that can be adopted and 
implemented directly by banks in Indonesia. This is due to the fact that the implementation of information 
technology in the banking industry in Indonesia is required to refer to the regulations issued by the regulator, 
in this case Bank Indonesia (BJ) and the financial services authority or known as Otoritas Jasa Keuangan 
(OJK). Some of the regulations issued by the regulator include the existence of on-premise obligations. On- 
Premise simply means that it must manage the infrastructure independently, both hardware, software, 
network, and so forth. In addition, the regulator also requires that the software used is the property of the 
Bank and complemented with at least two-factor authentication. Two-factor authentication is a security 
process where the user provides two means of identification, one of which is usually a physical sign, like a 
card, and the other is usually something memorized, such as a security code [2]. These things are challenge 
for banks in Indonesia for adopting Chatbot technology. These challenges are due to the fact that the existing 
Chatbot platforms can barely be implemented on-premise basis. In addition, most of several existing Chatbot 
platforms still use English as a knowledge base, thus the language used by the Chatbot platform is still one of 
the obstacles encountered considering that Indonesian is still the main language of communication in banking 
in Indonesia. 

Based on the explanation above, there are several services in banking that can be replaced by 
adopting Chatbot technology. This paper outlines a discussion related to the Chatbot platform that can meet 
the requirements of banking regulators in Indonesia. It can also use Indonesian as a knowledge base, and how 
the architecture needs to be adjusted to suit the process business and regulations in the bank industry in 
Indonesia. Additionally, a discussion on the evaluation of Chatbot platforms will also be elaborated and 
analyzed further. 


2. RESEARCH METHOD 
In a study, a method is needed. This method is used as a reference to get the expected results. The 
method used to complete this paper is shown in Figure 1. 


2.1. Methodological approach 
A methodological approach is an approach that will be taken to explore the topic. One 
methodological approach that is commonly used is a literature review. This phase is divided into 2 stages. 


2.1.1. Literature review 

In the literature study stage, it is carried out by collecting and reading writings in the form of articles 
and journals related to the research topic. Besides that, it can also be done by discussing with subject matter 
expert. As for the things that are learned including government regulation and Chatbot. 
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Figure 1. Research method 


a. Government regulation 

Currently, banks in Indonesia, in carrying out all their activities, must comply with all regulations 
issued by regulators, namely Bank Indonesia (BI) and the financial services authority Otoritas Jasa Keuangan 
(OJK). The two regulators, in drafting rules for banking, must refer to other rules defined by the Government 
of Indonesia. The most recent one is the rules regarding Electronic System and Transaction Management 
stated in the Government Regulation of the Republic of Indonesia peraturan pemerintah (PP) no. 71/2019. 
This PP has generally allowed private institutions to use cloud technology. However, it has not been 
specifically explained what data is allowed. 

The latest regulatory rules for the implementation of electronic transactions and systems and digital 
services in the banking industry are stated in the financial services authority regulation Peraturan Otoritas 
Jasa Keuangan (POJK) Number 12/POJK.03/2018 concerning the implementation of digital banking 
services, where the POJK does not refer to PP No. 71 of 2019. Broadly speaking, the POJK still requires 
every application of digital banking service to use the bank's software with the bank's hardware or the 
customer's or prospective customer's hardware. Unlike the case with Bank Indonesia Regulation Peraturan 
Bank Indonesia (PBI) Number 18/40/PBI/2016 concerning the implementation of payment transaction 
processing. In PBI, every application to be used by a customer or prospective customer must be registered 
and approved by Bank Indonesia, where in application registration or customer service, it is equipped with 
several supporting documents, including testing documents (SIT, UAT, and security testing). 


b. Chatbot 
Chatbot is a computer program designed to simulate an interactive conversation or communication 
with the user through text, voice, and visuals. Conversations that occur between computers and humans are a 
form of response from programs that have been declared in the program database on the computer. The 
resulting response is the result of scanning keywords in words or sentences from the user and produces the 
response that is considered the most suitable, or the pattern of words that are considered the closest from the 
database [3]-[12]. In everyday language, Chatbot is an application or computer program designed to imitate 
humans themselves. The limitation taken from Chatbot is that it can imitate human communication. Thus, if 
humans are chatting with the program, it is as if there are two humans communicating with each other. In 
fact, humans communicate with robots. The robot has been designed to respond to all kinds of questions and 
statements entered by users. This occurs because it has previously been declared in the database, in the form 
of word entities, sentence patterns, and various types of statements and questions [13]-[15]. The Chatbot 
component diagram is shown in Figure 2 [16]. 
The Chatbot software package consists of three components. The components are being as: 

— Responder. The responder is the interface that connects with the user and controls the input made by the 
user and the output of the replies. Data from responder will be sent to Classifier. 

— Classifier. Classifier functions to normalize and filter data sent from users. The data submitted by the 
user will then be replaced and divided into logical components. The classifier moves normalized 
sentences into the Graph-master component of the Chatbot. The classifier processes the output of the 
graph-master component. Apart from that, the classifier also handles database syntax instructions. 
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— Graph Master. The Chatbot graph-master component handles matching patterns. Graph-master is 
responsible for managing content storage or it can be said that Graph-master is the brain of Chatbot. 
The graph-master also handles the pattern matching process and pattern matching algorithms. 


Graph Master | | Classifier Responder ‘| User Interface 


Simulation Interface 


Figure 2. Chatbot component 


2.2. Development 

The development stage is a stage carried out by researchers to explore existing platforms. In this 
phase, the necessary data is collected and then the data is used to develop applications. There are three stages 
in this phase, these stages are being as. 


2.2.1. Development of platform botika 

Botika is a startup with a chatbot product or a smart chat assistant that has the task for answering 
user questions and complaints through the Facebook Messenger, LINE and Telegram chat applications. All 
messages that come through chat applications connected to Botika can be answered from one place. In 
addition, Botika can reply to messages for 24 hours using the Indonesian language and provide complete 
information about the products being sold. Hence, the role of Botika is to replace the role of customer service 
who has limited time to reply to messages from buyers. Besides making work easier, Botika can also reduce 
company costs for customer service. 

The Botika Platform is a chatbot platform whose environment is in a cloud, so that all conversation 
data that occurs on the chatbot will later be stored in the cloud for data analytic needs. This is one of the 
obstacles when implementing the Botika Platform in the banking industry in Indonesia, because the current 
regulatory regulations do not yet clearly state what data can be stored in the cloud. Develop a Chatbot 
application on the Botika Platform, user access to a dashboard that can be accessed via https://botika.online is 
required. After successfully logging into the dashboard, the next steps are creating the following aspects: 

— State. State is a setting that defines the position of a Chatbot. Thus, the Chatbot will know after which 
state it will go to which state. 

— Action. Action is a command that is executed by Chatbot in any predefined state where to add 
commands using the PHP programming language. 

— Transition. Transition is used to change the condition of a state with the aim that Chatbot can process or 
move to another state. 

— Integration. The Integration feature functions to integrate the bot into a social media or a web platform 
to be paired. Botika Platform already provides a feature to facilitate Chatbot integration to the Chat 
Platform on social media, for example WhatsApp, Facebook, Telegram, and so on. However, this 
research is only integrated into the webchat which is integrated into the Qwerty Bank corporate web. 

— Intent. The Intent feature on the Chatbot functions as a knowledge base, meaning that this feature is 
used to register questions along with their answers. To register, we can use the bulk method (csv file 
format) or one by one. 

— Train intent. Train intent is a feature on the Botika dashboard. This feature is used to carry out the 
training data process, after an intent is added, so that it can be recognized by the Bot. 

— Test Intent. This feature is used to assure that the data being trained can be recognized by the Chatbot 
through a string similarity approach. The probability level of a word or sentence delivered can be 
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measured from the Test Intents feature. For example, a user submits the word "Halao". The results can 

be seen in the Figure 3. 
In order for the Botika Platform to comply with regulatory regulations, additional development is required on 
top of the Botika dashboard. This is carried out for features where confidential data is needed, such as card 
number, identity number, and date of birth. This additional development is carried out by developing a web- 
based form created using PHP to send the results to a Qwerty Bank server. Afterwards, the Botika Platform 
will display a link to a form that will open once a customer clicks it. The form can be filled by customers or 
prospective customers. That way, confidential data will not be sent and stored on the Botika Platform but 
stored on servers in the Qwerty Bank Data Center. 


ASTRID says halao 


Intent sys.info_greeting 


Probability 0.3063 


Figure 3. Probability in Botika platform 


2.2.2. Development of rasa platform 

Rasa Core and Rasa natural language understanding (NLU) is an open sources Python library for 
making conversational software, with the aim of creating machine learning and language understanding 
based dialogue management that can be developed by non-specialist software developers. Rasa can be 
developed by environment on premise and using the Python programming language [17]-[23]. The Rasa 
Platform can be implemented in an on-premises environment but cannot yet be implemented in the banking 
industry in Indonesia, because the library currently used is spacy version 1.8.2, where this version cannot 
understand Indonesian perfectly. This is an obstacle when implementing the Rasa Platform. Develop a 
Chatbot application on the Rasa Platform, several supporting software are needed, including Anaconda3, 
Python3, Rasa Package version 1.8.1, and Apache. Such software is installed in one environment. The 
environment used has the specifications of 1 Core, 4 GB RAM, and 100 GB hard disk drive (HDD). After the 
supporting software are installed, the next steps are: 

— Intent. Intent is a question mapping that will be asked to the created a chatbot. The intent will later 
become the knowledge base for the chatbot application. To describe the intent on the Rasa Platform, it 
is saved in a file with .yml extension. 

— Config. Config is the process of configuring the components needed by the Chatbot. Each configuration 
is defined in a file and saved in .yml format. 

— Action. Action is a file containing programming in Python, where the file is saved in .py format. The 
action file is a collection of commands that will be executed by Chatbot, where the commands are based 
on the variables that have been registered in the domain file. The command can be in the form of 
response text or connect to a service or database. 

When viewed from the regulatory perspective, there is no rule violated when implementing the Rasa 
Platform in the banking industry in Indonesia, because there is no regulation from the regulator that requires 
applications to use Indonesian. However, the implementation of chatbot using the Rasa Platform is not quite 
right due to the fact that the majority of the Qwerty Bank’s customers are Indonesians, thus it would be more 
appropriate if the chatbot also uses Indonesian. For this reason, a modification to the Rasa Platform is carried 
out by replacing or adding relevant Python libraries, for example by adding the natural language toolkit 
(NLTK) library which is used for natural language processing such as classification, tokenization, stemming, 
tagging, and parsing. Moreover, a newer version (at least version 2.0) of the Spacy library is used since it 
already supports Indonesian. 


2.3. Evaluate and result 

In evaluate and result stage, a series of tests were carried out on the system that had been developed 
previously. This is performed to detect errors early, ensure the system's ability to run, and validate the 
correctness of the system. After testing is complete, the next step is to evaluate the results. 


2.3.1. Testing platform botika 

This test is carried out using the Blackbox testing method. Blackbox testing is a software testing 
method that examines application functionality without peering into its internal structure or workings. The 
results of these tests can be seen in Table 1. 
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Table 1. Test result from Botika Platform 


Feature Test Scenario Step Data Expected Results _ Result Pass/Fail 
1. Greeting 
2. Choose “Informasi Saldo” 
3. Click Form Card Number: Balance 
Account Display 4. Input Data (Debit Card Number, S29 EEAGS3 nate 
: : Information is Success Pass 
Balance Balance Date of Birth) Date of Birth: Displayed 
5. Click Submit D*/0#/L*2 piey 
6. Input OTP 
7. Click Submit 
1. Greeting 
2. Select saving information : 
«ce : Information on 
3. Choose “Tabungan : 
: Show Product . . : Savings Products 
Saving : 4. Savings information: - Success Pass 
Information Appears 
a. Feature and cost Successfull 
b. Facility ieee 
c. Benefit 


2.3.2. Testing rasa platform 

Blackbox testing is also used to perform testing on the Rasa platform. This is done so that the 
comparison results are comparable to the previous platform. For the Rasa platform test, the test results are as 
shown in Table 2. 


Table 2. Rest result from Rasa Platform 


Feature Test Scenario Step Data Expected Results Result Pass/Fail 
Account Display a Greeuiie F me Account Number: Balan oe 
Balance balance 2. Type “Informasi Saldo 430 1 #8987 Information is Success Pass 
3. Type Account Number Displayed 
Saving Show product 1. Greeting 
information 2. Type “Informasi Produk” : 
2 Fc Information on 
3. Choose “Tabungan ‘ 
é ; as Savings Products 
4. Savings information: - Success Pass 
Appears 
a. Feature and cost Successfull 
b. Facility Peg 
c. Benefit 


2.3.3. Evaluation of result 

The evaluation was carried out using the usability method and the hedonic motivation system 
adoption model (HMSAM) method. Both methods are used to obtain a value, where the value is obtained by 
giving a questionnaire to at least 15 correspondents with the aim that the results obtained were in accordance 
with predetermined goals and objectives. Usability consists of three components: 

—  Think-aloud evaluation (TA). Users are asked to have their opinion and express their feelings when 
interacting with the application. 

— Cognitive walkthrough (CW). The evaluator works through series of task scenarios and asks a number 
of questions from the user's perspective. 

— Heuristic evaluation (HE) is a principle used to evaluate the interaction and interface of a system or 
application [24], [25]. 

The HMSAM method is a model for measuring a system that adapts to hedonic motivation [26]. 
Hedonic motivation refers to the influence of a person's pleasure and pain receptors on their willingness to 
move towards a goal or away from a threat*. There are five factors that are the focus of measurement in 
HMSAM, namely perceived usefulness, perceived ease of use, curiosity, control, and joy, where these five 
factors will influence behavioral intention to use and immersion of an application [26]. Thus, HMSAM has 
five aspects that affect the evaluation results. These aspects are: 

— Perceived usefulness, used to measure the increase in performance when using a system. 

— Perceived ease of use, used to measure the ease of use of the system. 

— Curiosity, used to measure the extent to which a system can increase curiosity in cognitive aspects. 
— Control, used to measure the user's perception that he is being invited to interact by the system. 

— Joy, used to explore the pleasure aspects of users interacting with the system. 

To get the values of each usability and HMSAM component, it is performed by giving a 
questionnaire to at least 15 correspondents. The questionnaire contains 17 questions. The composition is 
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presented in Table 3. Correspondents were asked for their opinion regarding the Chatbot application by 
providing a choice of categories Strongly Agree (4), Agree (3), Disagree (2), and Strongly Disagree (1) for 
each question. 


Table 3. Composition question usability method and HMSAM method 
Component Method _ Quantity 


Think-Aloud Evaluation (TA) — Usability 5 
Cognitive Walkthrough (CW) Usability 6 
Heuristic Evaluation (HE) Usability 6 
Perceived ease of use HMSAM 4 
Perceived usefulness HMSAM 3 
Curiosity HMSAM 4 

Control HMSAM 4 

Joy HMSAM 3 


3. RESULTS AND DISCUSSION 

Based on the methodology previously described, this section discusses the results achieved. In terms 
of application, it produces two Chatbot applications with different platforms. To make the comparison of the 
two chatbot platforms more comprehensive, assessments obtained from the user's side were also conducted. 
Two methods were used: usability and the hedonic motivation system adoption model (HMSAM) method. 
To get the value of each component, a questionnaire method was used. The questionnaire consisted of 17 
statements which were asked to 30 respondents. The 17 statements cover the components of the usability 
method and the HMSAM. Table 4 describes the results of 30 respondents who filled out the questionnaire. 


Table 4. Questionnaire results of usability and HMSAM method 


No Component Component of Botika Rasa 
of Usability HMSAM Total Average Total Average 
1 HE Perceived ease of use 104 3.467 93 3.1 
2 TA Perceived ease of use 99 3.3 93 3.1 
3 HE Control 97 3.233 88 2.933 
4 HE Joy 92 3.067 83 2.767 
5 CW Perceived ease of use 101 3.367 93 3.1 
6 CW Curiosity 99 3.3 88 2.933 
7 TA Perceived ease of use 98 3.267 92 3.067 
8 HE Control 93 3.1 85 2.833 
9 CW Curiosity 107 3.567 99 3.3 
10 CW Joy 100 3.333 90 3 
11 HE Perceived usefulness 92 3.067 85 2.833 
12 TA Perceived usefulness 102 3.4 94 3.133 
13 TA Perceived usefulness 102 3.4 96 3.2 
14 HE Control 96 3.2 92 3.067 
15 TA Joy 92 3.067 89 2.967 
16 CW Control 89 2.967 83 2.767 
17 CW Curiosity 83 2.767 719 2.633 


3.1. Validity and reliability test of the questionnaire 

Whether a research instrument is good or not is determined by its validity and reliability. The 
validity of the instrument concerns the extent of measurement accuracy in measuring what you want to 
measure, while reliability questions the extent to which a measurement can be trusted because of its 
consistency. The instrument is said to be valid if it can reveal data from the variables appropriately not 
deviating from the actual situation. The instrument is said to be reliable if it can reveal reliable data [27]. In 
this study, the validity and reliability tests were carried out with the results showing that the T table value 
(5%, 30) was 1.7. The data is declared valid because the value of T count 17 statements is greater than the 
value of T table. To perform a reliability test, the variance value, the number of variance and the total 
variance are needed. The variants are obtained from the var function for all respondents’ answers in each 
statement. The total variance is obtained from the sum of all variance values. After obtaining these three 
values, then it is used to calculate the reliability value obtained by a value of 0.91. 


3.2. Usability method 
The results in Table 4 are then grouped by components, which are presented in Table 5. The Total 
value is obtained from the sum of the average values in Table 4 according to the components, where the 
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results of the total column will be divided by the number of questions according to the components, the 
results of the division are placed in the Average column. From Table 5, user ratings of the two Chatbot 
platforms prefer the Botika platform to the Rasa platform. This can be proven from the values in the Average 
and Percentage (%) columns of the two platforms tested. For the TA component, users rated the Botika 
platform with a score of 82.15% better than the Rasa platform which had a value of 77.325%. Furthermore, 
for the CW component, the Botika platform has a value of 80.4% while the Rasa platform has a value of 
78.875%. Finally, for the HE component, users also rated the Botika platform higher with a value of 79.7% 
when compared to the Rasa platform which was valued at 73.05%. 


Table 5. Result comparison of usability method 


Component of Usability Banks gia 
Total Average Percentage (%) Total Average __ Percentage (%) 
TA 16.433 3.286 82.15 15.466 3.093 77.325 
CW 19.3 3.216 80.4 17.733 2.955 73.875 
HE 19.133 3.188 79.7 17.533 2.922 73.05 


3.3. Hedonic motivation system adoption model (HMSAM) 

The results in Table 4 are then grouped by components, which are presented in Table 6. From Table 
6, it can be seen that users judge that the Botika platform has a higher value than the Rasa platform. It can be 
seen from Table 7 that all the HMSAM components, the Botika platform have a higher value. The Perceived 
ease of use component of the Botika platform is valued at 83.75%, while the Rasa platform has a value of 
77.275%. Furthermore, the perceived usefulness component is valued at 82.2% for the Botika platform and 
76,275 for the Rasa platform. Then, respectively, the curiosity, control, and joy components on the Botika 
platform compared to the Rasa platform were 80.275% compared to 73.875%, 78.125% with 72.5%, and 
78.875% compared to 72.775%. From Table 6, arithmetic calculations are then carried out to get the 
behavioral intention to use and immersion values as can be seen in Table 7. 


Table 6. Result Comparison of HMSAM Method 


Botika Rasa 
FMS ANE Con ponear Total Average Percentage(%) Total | Average Percentage (%) 
Perceived Ease of Use 13.4 3.35 83.75 12.366 3.091 771.275 
Perceived Usefulness 9.866 3.288 82.2 9.166 3.055 76.375 
Curiosity 9.633 3.211 80.275 8.866 2.955 73.875 
Control 12.5 3.125 78.125 11.6 2.9 72.5 
Joy 9.366 3.155 78.875 8.733 2.911 Wdd5 


Table 7. HMSAM behavioral invention to use dan immersion 


Botika Rasa 
Ease touse Usefulness Curiosity Joy Control Ease touse Usefulness Curiosity Joy Control 
3.35 3.288 3.211 3.155 3.125 3.091 3.055 2.955 2.911 2.9 
oe ee 6.638 6.561 6.505 6.475 6.146 6.046 6.002 5.991 
pyprage Bacelved 3.319 3.281 3.253 3.238 3.073 3.023 3.001 2.996 
ease to use 
Behavioral intention to use 
Total 9.852 9.097 
Average Behavioral 3.284 3.032 
Percentage 
Behavioral 82.1 ENP 
Immersion 
Total 9.771 9.020 
Average Immersion 3.257 3.007 
Percentage 81.421 75.163 
Immersion 


Table 7 shows that the percentage value of Behavioral intention to use is 82.1% for the Botika 
Platform and 75.808% for the Rasa Platform. This value is influenced by the perceived usefulness, curiosity, 
and joy values. As for the Immersion value, the Botika Platform is higher with a value of 81.421% compared 
to the Rasa Platform which is 75.163%. Immersion value is greatly influenced by the curiosity, joy, and 
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control components. By looking at the value of the Behavioral intention to use and Immersion components, it 

means that users will use Chatbot technology more when the Botika Platform is implemented rather than the 

Rasa Platform which is implemented. The data in Table 8 is then used to measure the significance test, 

namely "t-Test: two-sample assuming equal variances". The results are shown in Table 9. 

a. Hypothesis 

—  HO0:ul=w2 (There is no significant difference between the mean Botika and Rasa evaluations). 

—  H1:yl> 2 (There is a significant difference between the mean Botika and Rasa evaluations). 
Significance Level (a=5% or 0.05). 

c. Acceptance criteria 
Accept HO if t count=t table or p-value> alpha (a) or Reject HO If t count> t table or p-value=alpha (a). 

d. Interpretation of the results of statistical analysis hypothesis test t-test 

— Mean is the average value of Botika Output=3.239 and the average value of Rasa Output=2.993. 

— Variance is the value of the variation in Botika Output = 0.0054 and the value of the variation in the 
Rasa Output=0.0046. 

— Observations are the number of Botika and Rasa observations, each of which consists of 10 
observations. 

— Pooled Variance is a combined variation of Botika and Rasa, namely 0.0050. 

— Hypothesized Mean Difference is the average difference between Botika and Rasa, but for this case 
example, it is assumed that there is no difference so that the value is "0". 

—  dfis the Degree of Freedom or degrees of freedom obtained by calculating nl+n2-2 so that 8+8-2=14. 

—  t-stat is the value of t count which is equal to 6.951. 

—  tcritical one tail is t table value that is 1.761. 

— Because the hypothesis shows one direction, namely u1>p2 (bigger), then what is seen is only the p- 
value and t table (t critical) in one direction, namely ONE TAIL. 

e. Statistical conclusions 

—  tcount (6.951)>t Table (1.761) means we reject HO (ACCEPT H1), or 

— p-value (3.3743)>alpha (0.05) means we reject HO (ACCEPT H1). 

—  Botika is more suitable to be implemented based on the significant differences with the Rasa. 


Table 8. Average component HMSAM 


No HMSAM Component Botika Rasa 
1 Perceived ease of use 3.35 3.091 
2 Perceived usefulness 3.288 3.055 
3 Curiosity 3.211 2.955 
4 Control 3.125 2.9 
5 Joy 3.155 2.911 
6 Behavioral intention to use 3.284 3.032 
7 Immersion 3.257 3.007 
8 All Component 3.239 2.993 


Table 9. t-Test: Two-sample assuming equal variances 
t-Test: Two-Sample Assuming Equal Variances 


Variable 1 Variable 2 


Mean 3.238571429 2.993 
Variance 0.005415102 0.004568857 
Observations 8 8 
Pooled Variance 0.00499 198 
Hypothesized Mean Difference 0 
df 14 

t Stat 6.95 138643 
P(T<=t) one-tail 3.37433E-06 
t Critical one-tail 1.761310136 
P(T<=t) two-tail 6.74867E-06 
t Critical two-tail 2.144786688 


4. CONCLUSION 

The results of the two methods used to test, namely usability and HMSAM on the Botika Platform 
and Rasa Platform, show that users prefer the Botika Platform from all aspects of testing. These results also 
provide an overview of the expectations to be achieved in the study. The implementation of Chatbot 
technology at Qwerty Bank can at least provide new services that make it easier for customers and 
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prospective customers. In addition, by obtaining a Chatbot Platform that is suitable for Qwerty Bank, it is 
hoped that in the future all services at the call center can be implemented so that Qwerty Bank can make 
savings every month for Call Center fees of Rp 625,000,000.00. In addition, these results can also be used as 
a reference for banks in Indonesia to implement Chatbot technology in accordance with regulations. In the 
future, it is hoped that it can assist banking in Indonesia in implementing Chatbot Technology which can be 
used to provide easy service for its customers. In addition, in the future it is still possible for the Chatbot 
Platform to be implemented for other features or even replace the customer service and call center as a 
whole, thereby reducing operational costs. In addition, it can also be used as a platform to support the 
concept of digital banking and even branchless banking. 
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